Simultaneous Truth And Performance Level Estimation Research Articles

Evaluating the relationship between contouring variability and modelled treatment outcome for prostate bed radiotherapy.

Objectives.Contouring similarity metrics are often used in studies of inter-observer variation and automatic segmentation but do not provide an assessment of clinical impact. This study focused on post-prostatectomy radiotherapy and aimed to (1) identify if there is a relationship between variations in commonly used contouring similarity metrics and resulting dosimetry and (2) identify the variation in clinical target volume (CTV) contouring that significantly impacts dosimetry.Approach.The study retrospectively analysed CT scans of 10 patients from the TROG 08.03 RAVES trial. The CTV, rectum, and bladder were contoured independently by three experienced observers. Using these contours reference simultaneous truth and performance level estimation (STAPLE) volumes were established. Additional CTVs were generated using an atlas algorithm based on a single benchmark case with 42 manual contours. Volumetric-modulated arc therapy (VMAT) treatment plans were generated for the observer, atlas, and reference volumes. The dosimetry was evaluated using radiobiological metrics. Correlations between contouring similarity and dosimetry metrics were calculated using Spearman coefficient (Γ). To access impact of variations in planning target volume (PTV) margin, the STAPLE PTV was uniformly contracted and expanded, with plans created for each PTV volume. STAPLE dose-volume histograms (DVHs) were exported for plans generated based on the contracted/expanded volumes, and dose-volume metrics assessed.Mainresults. The study found no strong correlations between the considered similarity metrics and modelled outcomes. Moderate correlations (0.5 <Γ< 0.7) were observed for Dice similarity coefficient, Jaccard, and mean distance to agreement metrics and rectum toxicities. The observations of this study indicate a tendency for variations in CTV contraction/expansion below 5 mm to result in minor dosimetric impacts.Significance. Contouring similarity metrics must be used with caution when interpreting them as indicators of treatment plan variation. For post-prostatectomy VMAT patients, this work showed variations in contours with an expansion/contraction of less than 5 mm did not lead to notable dosimetric differences, this should be explored in a larger dataset to assess generalisability.

The role of 18F-FDG PET in minimizing variability in gross tumor volume delineation of soft tissue sarcomas

BackgroundAccurate gross tumor volume (GTV) delineation is a critical step in radiation therapy treatment planning. However, it is reader dependent and thus susceptible to intra- and inter-reader variability. GTV delineation of soft tissue sarcoma (STS) often relies on CT and MR images. PurposeThis study investigates the potential role of 18F-FDG PET in reducing intra- and inter-reader variability thereby improving reproducibility of GTV delineation in STS, without incurring additional costs or radiation exposure. Materials and MethodsThree readers performed independent GTV delineation of 61 patients with STS using first CT and MR followed by CT, MR, and 18F-FDG PET images. Each reader performed a total of six delineation trials, three trials per imaging modality group. Dice Similarity Coefficient (DSC) score and Hausdorff distance (HD) were used to assess both intra- and inter-reader variability using generated simultaneous truth and performance level estimation (STAPLE) GTVs as ground truth. Statistical analysis was performed using a Wilcoxon signed-ranked test. ResultsThere was a statistically significant decrease in both intra- and inter-reader variability in GTV delineation using CT, MR 18F-FDG PET images vs. CT and MR images. This was translated by an increase in the DSC score and a decrease in the HD for GTVs drawn from CT, MR and 18F-FDG PET images vs. GTVs drawn from CT and MR for all readers and across all three trials. ConclusionIncorporation of 18F-FDG PET into CT and MR images decreased intra- and inter-reader variability and subsequently increased reproducibility of GTV delineation in STS.

Validation of QyScore’s® fully automated quantitative image segmentation tools against expert manual gold‐standards

AbstractBackgroundQuantitative imaging provides valuable information for early detection, disease progression and treatment response monitoring. Manual segmentation is the current gold‐standard, but is prohibitively labour‐intensive for large scale use, such as in clinical routine, and suffers from individual variability. Thus, there is a substantial unmet need for validation of automatic segmentation techniques that perform as accurately as this labour‐intensive manual gold‐standard.MethodThe validation cohort consisted of 50 individuals with multiple diagnoses, wide age range (18‐86 yrs), balanced for sex and acquired on multiple scanners for improved generalizability of quantification results (Table 1). Three expert neuroradiologists each manually segmented the caudate, putamen, globus pallidus, thalamus, cerebellum, brainstem, and lateral ventricles on 3DT1 images using itk‐SNAP (http://www.itksnap.org/) software. A consensus expert segmentation was then derived using the STAPLE (Simultaneous Truth and Performance Level Estimation) algorithm (Wakefield et al., (2004)) for each region and compared with the fully automated segmentations produced using QyScore®, a CE‐marked and FDA‐cleared neuroimaging medical device. Performance was investigated using the Dice Similarity Coefficient (DSC) and concordance assessed with plotted linear regression.ResultMean, standard deviation and confidence intervals of the DSC demonstrated a high degree of agreement between the consensus manual gold‐standard and QyScore®’s automated segmentations (Table 2). Importantly, this agreement remained consistent following stratification by field strength, demonstrating generalizability to most clinical imaging centres. For 1.5T scanners the mean DSC was 0.81, 0.89, 0.79, 0.86, 0.94, 0.93 and 0.91 for the caudate, putamen, globus pallidus, thalamus, cerebellum, brainstem, and lateral ventricles respectively. For the 3T scanners, this was equivalent or marginally better at 0.84, 0.89, 0.82, 0.86, 0.94, 0.94, and 0.93 respectively. In addition, strong concordance was demonstrated between the volumes, expressed as a percentage of intracranial volume (%ICV), obtained by the automated medical device QyScore® and the consensus of the manual gold‐standard segmentation (Figure 1: Table 2).ConclusionQyScore® produces fast reliable automated segmentations with comparable accuracy to expert neuroradiologists. These findings support the implementation of QyScore® in clinical trials and in clinical routine to provide quantitative image analysis in support of diagnosis and monitoring of disease progression and treatment response.

Validation of QyScore’s® fully automated quantitative image segmentation tools against expert manual gold‐standards

AbstractBackgroundQuantitative imaging provides valuable information for early detection, disease progression and treatment response monitoring. Manual segmentation is the current gold‐standard, but is prohibitively labour‐intensive for large scale use, such as in clinical routine, and suffers from individual variability. Thus, there is a substantial unmet need for validation of automatic segmentation techniques that perform as accurately as this labour‐intensive manual gold‐standard.MethodThe validation cohort consisted of 50 individuals with multiple diagnoses, wide age range (18‐86 yrs), balanced for sex and acquired on multiple scanners for improved generalizability of quantification results (Table 1). Three expert neuroradiologists each manually segmented the caudate, putamen, globus pallidus, thalamus, cerebellum, brainstem, and lateral ventricles on 3DT1 images using itk‐SNAP (http://www.itksnap.org/) software. A consensus expert segmentation was then derived using the STAPLE (Simultaneous Truth and Performance Level Estimation) algorithm (Wakefield et al., (2004)) for each region and compared with the fully automated segmentations produced using QyScore®, a CE‐marked and FDA‐cleared neuroimaging medical device. Performance was investigated using the Dice Similarity Coefficient (DSC) and concordance assessed with plotted linear regression.ResultMean, standard deviation and confidence intervals of the DSC demonstrated a high degree of agreement between the consensus manual gold‐standard and QyScore®’s automated segmentations (Table 2). Importantly, this agreement remained consistent following stratification by field strength, demonstrating generalizability to most clinical imaging centres. For 1.5T scanners the mean DSC was 0.81, 0.89, 0.79, 0.86, 0.94, 0.93 and 0.91 for the caudate, putamen, globus pallidus, thalamus, cerebellum, brainstem, and lateral ventricles respectively. For the 3T scanners, this was equivalent or marginally better at 0.84, 0.89, 0.82, 0.86, 0.94, 0.94, and 0.93 respectively. In addition, strong concordance was demonstrated between the volumes, expressed as a percentage of intracranial volume (%ICV), obtained by the automated medical device QyScore® and the consensus of the manual gold‐standard segmentation (Figure 1: Table 2).ConclusionQyScore® produces fast reliable automated segmentations with comparable accuracy to expert neuroradiologists. These findings support the implementation of QyScore® in clinical trials and in clinical routine to provide quantitative image analysis in support of diagnosis and monitoring of disease progression and treatment response.

Reference standard for the evaluation of automatic segmentation algorithms: Quantification of inter observer variability of manual delineation of prostate contour on MRI

PurposeThe purpose of this study was to investigate the relationship between inter-reader variability in manual prostate contour segmentation on magnetic resonance imaging (MRI) examinations and determine the optimal number of readers required to establish a reliable reference standard. Materials and methodsSeven radiologists with various experiences independently performed manual segmentation of the prostate contour (whole-gland [WG] and transition zone [TZ]) on 40 prostate MRI examinations obtained in 40 patients. Inter-reader variability in prostate contour delineations was estimated using standard metrics (Dice similarity coefficient [DSC], Hausdorff distance and volume-based metrics). The impact of the number of readers (from two to seven) on segmentation variability was assessed using pairwise metrics (consistency) and metrics with respect to a reference segmentation (conformity), obtained either with majority voting or simultaneous truth and performance level estimation (STAPLE) algorithm. ResultsThe average segmentation DSC for two readers in pairwise comparison was 0.919 for WG and 0.876 for TZ. Variability decreased with the number of readers: the interquartile ranges of the DSC were 0.076 (WG) / 0.021 (TZ) for configurations with two readers, 0.005 (WG) / 0.012 (TZ) for configurations with three readers, and 0.002 (WG) / 0.0037 (TZ) for configurations with six readers. The interquartile range decreased slightly faster between two and three readers than between three and six readers. When using consensus methods, variability often reached its minimum with three readers (with STAPLE, DSC = 0.96 [range: 0.945–0.971] for WG and DSC = 0.94 [range: 0.912–0.957] for TZ, and interquartile range was minimal for configurations with three readers. ConclusionThe number of readers affects the inter-reader variability, in terms of inter-reader consistency and conformity to a reference. Variability is minimal for three readers, or three readers represent a tipping point in the variability evolution, with both pairwise-based metrics or metrics with respect to a reference. Accordingly, three readers may represent an optimal number to determine references for artificial intelligence applications.

Consensus Delineation Guidelines for Pelvic Lymph Node Radiation Therapy of Prostate Cancer: On Behalf of the Francophone Group of Urological Radiation Therapy (GFRU)

Clinical target volume (CTV) delineation for pelvic lymph nodes in prostate cancer is currently based on 3 consensus guidelines with some inherent discrepancies. To improve the reproducibility in nodal delineation, the Francophone Group of Urological Radiotherapy (Groupe Francophone de Radiothérapie Urologique [GFRU]) worked toward proposing an easily applicable, reproducible, and practice-validated contouring guideline for pelvic nodal CTV. The nodal CTV data sets of a high-risk node-negative prostate cancer clinical case contoured by 86 radiation oncologists participating in a GFRU contouring workshop were analyzed. CTV volumes were defined before and after a structured presentation of literature data on lymphatic drainage pathways and patterns of nodal involvement and relapse, illustrated using a reference contour (CRef) defined by 3 GFRU experts. The consistency between the participants' contours and CRef was assessed quantitively by means of the Simultaneous Truth and Performance Level Estimation (STAPLE) method, the Dice coefficient, and the Hausdorff distance and qualitatively using a count map. These results combined with the literature review were thoroughly discussed among GFRU experts to reach a consensus. From the 86 workshop participants, the volume of the STAPLE CTV was 591 cc compared with 502 cc for CRef. The Dice coefficient of the STAPLE CTV compared with the experts' CRef was 0.736 (±0.084) before and 0.823 (±0.070) after the workshop; the standard deviation decreased from 11.5% to 8.5% over the workshop. The Hausdorff distance of the STAPLE CTV compared with the CRef was 34.5 mm (±12.4) before the workshop and 21.8 mm (±9.3) after the workshop. Four areas of significant interobserver variability were identified, and a consensus was reached. Using a robust methodology, our cooperative group proposed an easily applicable, reproducible, and practice-validated guideline for the delineation of the pelvic CTV in prostate cancer, useful for implementation in daily practice and clinical trials.

Investigation of autosegmentation techniques on T2-weighted MRI for off-line dose reconstruction in MR-linac workflow for head and neck cancers.

In order to accurately accumulate delivered dose for head and neck cancer patients treated with the Adapt to Position workflow on the 1.5T magnetic resonance imaging (MRI)-linear accelerator (MR-linac), the low-resolution T2-weighted MRIs used for daily setup must be segmented to enable reconstruction of the delivered dose at each fraction. In this pilot study, we evaluate various autosegmentation methods for head and neck organs at risk (OARs) on on-board setup MRIs from the MR-linac for off-line reconstruction of delivered dose. Seven OARs (parotid glands, submandibular glands, mandible, spinal cord, and brainstem) were contoured on 43 images by seven observers each. Ground truth contours were generated using a simultaneous truth and performance level estimation (STAPLE) algorithm. Twenty total autosegmentation methods were evaluated in ADMIRE: 1-9) atlas-based autosegmentation using a population atlas library (PAL) of 5/10/15 patients with STAPLE, patch fusion (PF), random forest (RF) for label fusion; 10-19) autosegmentation using images from a patient's 1-4 prior fractions (individualized patient prior [IPP]) using STAPLE/PF/RF; 20) deep learning (DL) (3D ResUNet trained on 43 ground truth structure sets plus 45 contoured by one observer). Execution time was measured for each method. Autosegmented structures were compared to ground truth structures using the Dice similarity coefficient, mean surface distance (MSD), Hausdorff distance (HD), and Jaccard index (JI). For each metric and OAR, performance was compared to the inter-observer variability using Dunn's test with control. Methods were compared pairwise using the Steel-Dwass test for each metric pooled across all OARs. Further dosimetric analysis was performed on three high-performing autosegmentation methods (DL, IPP with RF and 4 fractions [IPP_RF_4], IPP with 1 fraction [IPP_1]), and one low-performing (PAL with STAPLE and 5 atlases [PAL_ST_5]). For five patients, delivered doses from clinical plans were recalculated on setup images with ground truth and autosegmented structure sets. Differences in maximum and mean dose to each structure between the ground truth and autosegmented structures were calculated and correlated with geometric metrics. DL and IPP methods performed best overall, all significantly outperforming inter-observer variability and with no significant difference between methods in pairwise comparison. PAL methods performed worst overall; most were not significantly different from the inter-observer variability or from each other. DL was the fastest method (33s per case) and PAL methods the slowest (3.7-13.8 min per case). Execution time increased with a number of prior fractions/atlases for IPP and PAL. For DL, IPP_1, and IPP_RF_4, the majority (95%) of dose differences were within±250cGy from ground truth, but outlier differences up to 785cGy occurred. Dose differences were much higher for PAL_ST_5, with outlier differences up to 1920cGy. Dose differences showed weak but significant correlations with all geometric metrics (R2 between 0.030 and 0.314). The autosegmentation methods offering the best combination of performance and execution time are DL and IPP_1. Dose reconstruction on on-board T2-weighted MRIs is feasible with autosegmented structures with minimal dosimetric variation from ground truth, but contours should be visually inspected prior to dose reconstruction in an end-to-end dose accumulation workflow.

Information fusion for fully automated segmentation of head and neck tumors from PET and CT images.

PET/CT images combining anatomic and metabolic data provide complementary information that can improve clinical task performance. PET image segmentation algorithms exploiting the multi-modal information available are still lacking. Our study aimed to assess the performance of PET and CT image fusion for gross tumor volume (GTV) segmentations of head and neck cancers (HNCs) utilizing conventional, deep learning (DL), and output-level voting-based fusions. The current study is based on a total of 328 histologically confirmed HNCs from six different centers. The images were automatically cropped to a 200 × 200 head and neck region box, and CT and PET images were normalized for further processing. Eighteen conventional image-level fusions were implemented. In addition, a modified U2-Net architecture as DL fusion model baseline was used. Three different input, layer, and decision-level information fusions were used. Simultaneous truth and performance level estimation (STAPLE) and majority voting to merge different segmentation outputs (from PET and image-level and network-level fusions), that is, output-level information fusion (voting-based fusions) were employed. Different networks were trained in a 2D manner with a batch size of 64. Twenty percent of the dataset with stratification concerning the centers (20% in each center) were used for final result reporting. Different standard segmentation metrics and conventional PET metrics, such as SUV, were calculated. In single modalities, PET had a reasonable performance with a Dice score of 0.77±0.09, while CT did not perform acceptably and reached a Dice score of only 0.38±0.22. Conventional fusion algorithms obtained a Dice score range of [0.76-0.81] with guided-filter-based context enhancement (GFCE) at the low-end, and anisotropic diffusion and Karhunen-Loeve transform fusion (ADF), multi-resolution singular value decomposition (MSVD), and multi-level image decomposition based on latent low-rank representation (MDLatLRR) at the high-end. All DL fusion models achieved Dice scores of 0.80. Output-level voting-based models outperformed all other models, achieving superior results with a Dice score of 0.84 for Majority_ImgFus, Majority_All, and Majority_Fast. A mean error of almost zero was achieved for all fusions using SUVpeak , SUVmean and SUVmedian . PET/CT information fusion adds significant value to segmentation tasks, considerably outperforming PET-only and CT-only methods. In addition, both conventional image-level and DL fusions achieve competitive results. Meanwhile, output-level voting-based fusion using majority voting of several algorithms results in statistically significant improvements in the segmentation of HNC.

Chákṣu: A glaucoma specific fundus image database

We introduce Chákṣu–a retinal fundus image database for the evaluation of computer-assisted glaucoma prescreening techniques. The database contains 1345 color fundus images acquired using three brands of commercially available fundus cameras. Each image is provided with the outlines for the optic disc (OD) and optic cup (OC) using smooth closed contours and a decision of normal versus glaucomatous by five expert ophthalmologists. In addition, segmentation ground-truths of the OD and OC are provided by fusing the expert annotations using the mean, median, majority, and Simultaneous Truth and Performance Level Estimation (STAPLE) algorithm. The performance indices show that the ground-truth agreement with the experts is the best with STAPLE algorithm, followed by majority, median, and mean. The vertical, horizontal, and area cup-to-disc ratios are provided based on the expert annotations. Image-wise glaucoma decisions are also provided based on majority voting among the experts. Chákṣu is the largest Indian-ethnicity-specific fundus image database with expert annotations and would aid in the development of artificial intelligence based glaucoma diagnostics.

Assessing Inter-Annotator Agreement for Medical Image Segmentation.

Artificial Intelligence (AI)-based medical computer vision algorithm training and evaluations depend on annotations and labeling. However, variability between expert annotators introduces noise in training data that can adversely impact the performance of AI algorithms. This study aims to assess, illustrate and interpret the inter-annotator agreement among multiple expert annotators when segmenting the same lesion(s)/abnormalities on medical images. We propose the use of three metrics for the qualitative and quantitative assessment of inter-annotator agreement: 1) use of a common agreement heatmap and a ranking agreement heatmap; 2) use of the extended Cohen's kappa and Fleiss' kappa coefficients for a quantitative evaluation and interpretation of inter-annotator reliability; and 3) use of the Simultaneous Truth and Performance Level Estimation (STAPLE) algorithm, as a parallel step, to generate ground truth for training AI models and compute Intersection over Union (IoU), sensitivity, and specificity to assess the inter-annotator reliability and variability. Experiments are performed on two datasets, namely cervical colposcopy images from 30 patients and chest X-ray images from 336 tuberculosis (TB) patients, to demonstrate the consistency of inter-annotator reliability assessment and the importance of combining different metrics to avoid bias assessment.

Prospective imaging comparison of anatomic delineation with rapid kV cone beam CT on a novel ring gantry radiotherapy device

IntroductionA kV imager coupled to a novel, ring-gantry radiotherapy system offers improved on-board kV-cone-beam computed tomography (CBCT) acquisition time (17–40 seconds) and image quality, which may improve CT radiotherapy image-guidance and enable online adaptive radiotherapy. We evaluated whether inter-observer contour variability over various anatomic structures was non-inferior using a novel ring gantry kV-CBCT (RG-CBCT) imager as compared to diagnostic-quality simulation CT (simCT). Materials/methodsSeven patients undergoing radiotherapy were imaged with the RG-CBCT system at breath hold (BH) and/or free breathing (FB) for various disease sites on a prospective imaging study. Anatomy was independently contoured by seven radiation oncologists on: 1. SimCT 2. Standard C-arm kV-CBCT (CA-CBCT), and 3. Novel RG-CBCT at FB and BH. Inter-observer contour variability was evaluated by computing simultaneous truth and performance level estimation (STAPLE) consensus contours, then computing average symmetric surface distance (ASSD) and Dice similarity coefficient (DSC) between individual raters and consensus contours for comparison across image types. ResultsAcross 7 patients, 18 organs-at-risk (OARs) were evaluated on 27 image sets. Both BH and FB RG-CBCT were non-inferior to simCT for inter-observer delineation variability across all OARs and patients by ASSD analysis (p < 0.001), whereas CA-CBCT was not (p = 0.923). RG-CBCT (FB and BH) also remained non-inferior for abdomen and breast subsites compared to simCT on ASSD analysis (p < 0.025). On DSC comparison, neither RG-CBCT nor CA-CBCT were non-inferior to simCT for all sites (p > 0.025). ConclusionsInter-observer ability to delineate OARs using novel RG-CBCT images was non-inferior to simCT by the ASSD criterion but not DSC criterion.

PP09  Presentation Time: 10:50 AM: Interobserver Variability of Prostate MRI-Assisted Radiosurgery Postimplant Dosimetry: Comparison with Automatic Postimplant Dosimetry

<h3>Purpose</h3> To quantify the 2 sources of human interobserver variability (IoV) introduced during prostate MRI-assisted radiosurgery postimplant dosimetry, and compare this human variability to automatically generated postimplant dosimetry. <h3>Materials and Methods</h3> Twenty-five patients underwent LDR prostate brachytherapy (LDRPBT) and were imaged with fully balanced SSFP for postimplant dosimetry. Four certified medical dosimetrists (CMDs) identified the radioactive seeds in each of the 25 postimplant MRIs. Seven additional clinical observers, including 3 radiation oncologists (ROs), contoured the prostate and 4 organs at risk (OARs) on the MRIs. Consensus segmentation masks from the 3 ROs' contours were computed for each patient using the simultaneous truth and performance level estimation (STAPLE) algorithm. Thirty unique postimplant dosimetry plans were computed for each patient. Twenty-eight of the dosimetry plans were computed from each unique pair of the CMDs' seed plans and observer contours (4 CMDs × 7 contourers). One dosimetry plan was performed automatically for each patient using custom autosegmentation and automatic seed identification software developed previously. A reference dosimetry plan for each patient was computed using the consensus STAPLE segmentations and the seeds identified by the most experienced CMD. IoV of 5 dosimetry parameters was quantified separately for each IoV source: IoV introduced by seed localization, and IoV introduced by anatomy contouring. The coefficient of variation (CoV) of each dosimetry parameter was quantified for each patient across CMDs using the consensus STAPLE segmentations. Similarly, the CoV of each dosimetry parameter was quantified for each patient across contouring observers using the seed locations determined by the most experienced CMD. Additionally, the differences in dosimetry parameters between the reference and automatic dosimetry plans were quantified. <h3>Results/Discussion</h3> CoVs due to variability of anatomy contouring (CoV_contours) were significantly higher than those due to variability of seed localization (CoV_seeds) for all dosimetry parameters (median CoV_contours vs. median CoV_seeds: prostate D90 - 15.116% vs. 0.651%, p<0.001; prostate V100 - 5.359% vs. 0.365%, p<0.001; prostate V150 - 7.017% vs. 1.252%, p<0.001; rectum V100 - 79.230% vs. 8.688%, p<0.001; EUS V200 - 107.740% vs. 21.176%, p<0.001). CoV_contours were lower when the contouring observers were restricted to only the 3 ROs, but they were still significantly higher than CoV_seeds for all dosimetry parameters. Median differences in prostate D90, prostate V100, prostate V150, rectum V100, and EUS V200 between automatically computed and reference dosimetry parameters were 3.156%, 1.630%, -1.071%, -0.001%, and -0.002%, respectively. Most automatically computed dosimetry parameters were within the expected variability of those produced by human observers (Figure 1). <h3>Conclusion</h3> IoV of contouring constitutes the largest source of IoV in postimplant dosimetry plans performed by clinical observers. When using automatic contouring and seed localization tools, review and refinement of computer predictions will be necessary for some patients. However, automatic dosimetry tools can estimate postimplant dosimetry that is close to, or within, the expected variability of dosimetry plans produced by clinical observers.

GEC-ESTRO (ACROP)–ABS–CBG Consensus Brachytherapy Target Definition Guidelines for Recurrent Endometrial and Cervical Tumors in the Vagina

Representatives from the Gynecologic Groupe European de Curietherapie-European Society for Radiation Therapy and Oncology (GYN GEC-ESTRO), the American Brachytherapy Society (ABS), and the Canadian Brachytherapy Group (CBG) met to develop international consensus recommendations for target definitions for image-guided adaptive brachytherapy for vaginal recurrences of endometrial or cervical cancer. Seventeen radiation oncologists and 2 medical physicists participated. Before an in-person meeting each participant anonymously contoured 3 recurrent endometrial/cervical cancer cases. Participants contoured the residual gross primary tumor volume (GTV-Tres), a high-risk clinical target volume (CTV-THR), and an intermediate-risk clinical target volume (CTV-TIR), on T2-weighted magnetic resonance images (MRIs). All contours were drawn using Falcon EduCase. Contours were reviewed at an in-person meeting during which a consensus document was created defining agreed-upon target definitions (Trial 1). After establishing these definitions, the group was sent one of the cases again (recurrent cervical cancer vaginal recurrence) and asked to contour the targets again (Trial 2). The Computerized Environment for Radiation Research (CERR) software (The Mathworks, Natwick, MA) was used to analyze the contours. Kappa statistics were generated to assess level of agreement between contours. A conformity index (CI), defined as the ratio between the intersection and union volume of a given pair of contours, was calculated. A simultaneous truth and performance level estimation (STAPLE) contour was created for the CTV-THR and CTV-TIR for the postmeeting case. Consensus definitions for GTV-Tres, CTV-THR, and CTV-TIR were established. Kappa statistics (Trial 1/Trial 2) for GTV-Tres, CTV-THR, and CTV-TIR were 0.536/0.583, 0.575/0.743 and 0.522/0.707. Kappa statistics for Trial 2 for the CTV-THR and CTV-TIR showed "substantial" agreement while the GTV-Tres remained at moderate agreement. This consensus provides recommendations to facilitate future collaborations for MRI-guided adaptive brachytherapy target definitions in endometrial/cervical vaginal recurrences.

Impact of SSTR PET on Inter-Observer Variability of Target Delineation of Meningioma and the Possibility of Using Threshold-Based Segmentations in Radiation Oncology.

Simple SummaryDifferences in tumor segmentations between radiation oncologists is one of the largest sources of uncertainty in radiation therapy planning. This study investigated the influence of additional functional information from somatostatin receptor PET imaging on the inter observer variability in the delineation of meningioma. Further, this study assessed the usability of a simple thresholding approach for lesion delineation. It could be shown, that additional PET information was able to significantly reduce the inter observer variability. The threshold based delineation approach required a relatively low threshold value and showed only moderate agreement with the radiation oncologists.Aim: The aim of this study was to assess the effects of including somatostatin receptor agonist (SSTR) PET imaging in meningioma radiotherapy planning by means of changes in inter-observer variability (IOV). Further, the possibility of using threshold-based delineation approaches for semiautomatic tumor volume definition was assessed. Patients and Methods: Sixteen patients with meningioma undergoing fractionated radiotherapy were delineated by five radiation oncologists. IOV was calculated by comparing each delineation to a consensus delineation, based on the simultaneous truth and performance level estimation (STAPLE) algorithm. The consensus delineation was used to adapt a threshold-based delineation, based on a maximization of the mean Dice coefficient. To test the threshold-based approach, seven patients with SSTR-positive meningioma were additionally evaluated as a validation group. Results: The average Dice coefficients for delineations based on MRI alone was 0.84 ± 0.12. For delineation based on MRI + PET, a significantly higher dice coefficient of 0.87 ± 0.08 was found (p < 0.001). The Hausdorff distance decreased from 10.96 ± 11.98 mm to 8.83 ± 12.21 mm (p < 0.001) when adding PET for the lesion delineation. The best threshold value for a threshold-based delineation was found to be 14.0% of the SUVmax, with an average Dice coefficient of 0.50 ± 0.19 compared to the consensus delineation. In the validation cohort, a Dice coefficient of 0.56 ± 0.29 and a Hausdorff coefficient of 27.15 ± 21.54 mm were found for the threshold-based approach. Conclusions: SSTR-PET added to standard imaging with CT and MRI reduces the IOV in radiotherapy planning for patients with meningioma. When using a threshold-based approach for PET-based delineation of meningioma, a relatively low threshold of 14.0% of the SUVmax was found to provide the best agreement with a consensus delineation.

Variation in clinical target volume delineation in postoperative radiotherapy for biliary tract cancer.

We aimed to evaluate the inter-clinician variability in the clinical target volume (CTV) for postoperative radiotherapy (PORT) for biliary tract cancer (BTC) including extrahepatic bile duct cancer (EBDC) and gallbladder cancer (GBC). Nine experienced radiation oncologists delineated PORT CTVs for distal EBDC (pT2N1), proximal EBDC (pT2bN1) and GBC (pT2bN1) patients. The expectation maximization algorithm for Simultaneous Truth and Performance Level Estimation (STAPLE) was used to quantify expert agreements. We generated volumes with a confidence level of 80% to compare the maximum distance to each CTV in six directions. The degree of agreement was moderate; overall kappa values were 0.573 for distal EBDC, 0.513 for proximal EBDC, and 0.511 for GBC. In the distal EBDC, a larger variation was noted in the right, post, and inferior direction. In the proximal EBDC, all borders except the right and left direction showed a larger variation. In the GBC, a larger variation was found in the anterior, posterior, and inferior direction. The posterior and inferior borders were the common area having discrepancies, associated with the insufficient coverage of the paraaortic node. A consensus guideline is needed to reduce inter-clinician variability in the CTVs and adequate coverage of regional lymph node area.

Delineation uncertainties of tumour volumes on MRI of head and neck cancer patients

BackgroundDuring the last decade, radiotherapy using MR Linac has gone from research to clinical implementation for different cancer locations. For head and neck cancer (HNC), target delineation based only on MR images is not yet standard, and the utilisation of MRI instead of PET/CT in radiotherapy planning is not well established. We aimed to analyse the inter-observer variation (IOV) in delineating GTV (gross tumour volume) on MR images only for patients with HNC. Material/methods32 HNC patients from two independent departments were included. Four clinical oncologists from Denmark and four radiation oncologists from Australia had independently contoured primary tumour GTVs (GTV-T) and nodal GTVs (GTV-N) on T2-weighted MR images obtained at the time of treatment planning. Observers were provided with sets of images, delineation guidelines and patient synopsis. Simultaneous truth and performance level estimation (STAPLE) reference volumes were generated for each structure using all observer contours. The IOV was assessed using the DICE Similarity Coefficient (DSC) and mean absolute surface distance (MASD). Results32 GTV-Ts and 68 GTV-Ns were contoured per observer. The median MASD for GTV-Ts and GTV-Ns across all patients was 0.17 cm (range 0.08–0.39 cm) and 0.07 cm (range 0.04–0.33 cm), respectively. Median DSC relative to a STAPLE volume for GTV-Ts and GTV-Ns across all patients were 0.73 and 0.76, respectively. A significant correlation was seen between median DSCs and median volumes of GTV-Ts (Spearman correlation coefficient 0.76, p < 0.001) and of GTV-Ns (Spearman correlation coefficient 0.55, p < 0.001). ConclusionContouring GTVs in patients with HNC on MRI showed that the median IOV for GTV-T and GTV-N was below 2 mm, based on observes from two separate radiation departments. However, there are still specific regions in tumours that are difficult to resolve as either malignant tissue or oedema that potentially could be improved by further training in MR-only delineation.

Printed and Flexible ECG Electrodes Attached to the Steering Wheel for Continuous Health Monitoring during Driving.

Continuous health monitoring in a vehicle enables the earlier detection of symptoms of cardiovascular diseases. In this work, we designed flexible and thin electrodes made of polyurethane for long-term electrocardiogram (ECG) monitoring while driving. We determined the time for reliable ECG recording to evaluate the effectiveness of the electrodes. We recorded data from 19 subjects under four scenarios: rest, city, highway, and rural. The recording time was five min for rest and 15 min for the other scenarios. The total recording (950 min) is publicly available under a CC BY-ND 4.0 license. We used the simultaneous truth and performance level estimation (STAPLE) algorithm to detect the position of R-waves. Then, we derived the RR intervals to compare the estimated heart rate with the ground truth, which we obtained from ECG electrodes on the chest. We calculated the signal-to-noise ratio (SNR) and averaged it for the different scenarios. Highway had the lowest SNR (−6.69 dB) and rural had the highest (−6.80 dB). The usable time of the steering wheel was 42.46% (city), 46.67% (highway), and 47.72% (rural). This indicates that steering-wheel-based ECG recording is feasible and delivers reliable recordings from about 45.62% of the driving time. In summary, the developed electrodes allow continuous in-vehicle heart rate monitoring, and our publicly available recordings provide the opportunity to apply more sophisticated data analytics.

Automated segmentation of magnetic resonance bone marrow signal: a feasibility study

BackgroundManual assessment of bone marrow signal is time-consuming and requires meticulous standardisation to secure adequate precision of findings.ObjectiveWe examined the feasibility of using deep learning for automated segmentation of bone marrow signal in children and adolescents.Materials and methodsWe selected knee images from 95 whole-body MRI examinations of healthy individuals and of children with chronic non-bacterial osteomyelitis, ages 6–18 years, in a longitudinal prospective multi-centre study cohort. Bone marrow signal on T2-weighted Dixon water-only images was divided into three color-coded intensity-levels: 1 = slightly increased; 2 = mildly increased; 3 = moderately to highly increased, up to fluid-like signal. We trained a convolutional neural network on 85 examinations to perform bone marrow segmentation. Four readers manually segmented a test set of 10 examinations and calculated ground truth using simultaneous truth and performance level estimation (STAPLE). We evaluated model and rater performance through Dice similarity coefficient and in consensus.ResultsConsensus score of model performance showed acceptable results for all but one examination. Model performance and reader agreement had highest scores for level-1 signal (median Dice 0.68) and lowest scores for level-3 signal (median Dice 0.40), particularly in examinations where this signal was sparse.ConclusionIt is feasible to develop a deep-learning-based model for automated segmentation of bone marrow signal in children and adolescents. Our model performed poorest for the highest signal intensity in examinations where this signal was sparse. Further improvement requires training on larger and more balanced datasets and validation against ground truth, which should be established by radiologists from several institutions in consensus.

Variability of gross tumour volume delineation: MRI and CT based tumour and lymph node delineation for lung radiotherapy

PurposeTo compare gross tumour volume (GTV) delineation of lung cancer on magnetic resonance imaging (MRI) and positron emission tomography (PET) versus computed tomography (CT) and PET. MethodsThree experienced thoracic radiation oncologists delineated GTVs on twenty-six patients with lung cancer, based on CT registered to PET, T2-weighted MRI registered to PET and T1-weighted MRI registered with PET. All observers underwent education on reviewing T1 and T2 images along with guidance on window and level setup. Interobserver and intermodality variation was performed based on dice similarity coefficient (DSC), Hausdorff distance (HD), and average Hausdorff distance (AvgHD) metrics. To compute interobserver variability (IOV) a simultaneous truth and performance level estimation (STAPLE) volume for each image modality was used as reference volume. For intermodality analysis, each observers CT based primary and nodal GTV was used as reference volume. ResultsA mean DSC of 0.9 across all observers for primary GTV (GTVp) and a DSC of >0.7 for nodal GTV (GTVn) was demonstrated for IOV. Mean T2 and T1 GTVp and GTVn were smaller than CT GTVp and GTVn but the difference in volume between modalities was not statistically significant. Significant difference (p < 0.01) for GTVp and GTVn was found between T2 and T1 GTVp and GTVn compared to CT GTVp and GTVn based on DSC metrics. Large variation in volume similarity was noted based on HD of up-to 5.4 cm for observer volumes compared to STAPLE volume. ConclusionInterobserver variability in GTV delineation was similar for MRI and PET versus CT and PET. The significant difference between MRI compared to CT delineated volumes needs to be further explored.

Variability assessment of manual segmentations of ischemic lesion volume on 24-h non-contrast CT.

Infarct lesion volume (ILV) may serve as an imaging biomarker for clinical outcomes in the early post-treatment stage in patients with acute ischemic stroke. The aim of this study was to evaluate the inter- and intra-rater reliability of manual segmentation of ILV on follow-up non-contrast CT (NCCT) scans. Fifty patients from the Prove-IT study were randomly selected for this analysis. Three raters manually segmented ILV on 24-h NCCT scans, slice by slice, three times. The reference standard for ILV was generated by the Simultaneous Truth And Performance Level estimation (STAPLE) algorithm. Intra- and inter-rater reliability was evaluated, using metrics of intraclass correlation coefficient (ICC) regarding lesion volume and the Dice similarity coefficient (DSC). Median age of the 50 subjects included was 74.5years (interquartile range [IQR] 67-80), 54% were women, median baseline National Institutes of Health Stroke Scale was 18 (IQR 11-22), median baseline ASPECTS was 9 (IQR 6-10). The mean reference standard ILV was 92.5ml (standard deviation (SD) ± 100.9ml). The manually segmented ILV ranged from 88.2 ± 91.5 to 135.5 ± 119.9ml (means referring to the variation between readers, SD within readers). Inter-rater ICC was 0.83 (95%CI: 0.76-0.88); intra-rater ICC ranged from 0.85 (95%CI: 0.72-0.92) to 0.95 (95%CI: 0.91-0.97). The mean DSC among the three readers ranged from 65.5 ± 22.9 to 76.4 ± 17.1% and the mean overall DSC was 72.8 ± 23.0%. Manual ILV measurements on follow-up CT scans are reliable to measure the radiological outcome despite some variability.